THE GENERATING FUNCTION OF TWO-STACK 
SORTABLE PERMUTATIONS BY DESCENTS IS 
REAL-ROOTED 



PETTER BRANDEN 



Abstract. Bona has conjectured that, for fixed n > and t > 1, 
the generating function of t-stack sortable permutations of length n by 
descents is real-rooted. The conjecture is known to be true for t = 1 
and t — n — 1. Here we prove it for t — 2. 

1. Introduction 

Let Wt(n,k) be the number of t-stack sortable permutations in the sym- 
metric group, S n , with k descents. Recently, Bona E| showed that for 
fixed n and t the numbers Wt(n, k) form a unimodal sequence. Let W Ut t(x) = 
X^fc=o Wt(n, k)x k . A sufficient condition on a nonnegative sequence to be uni- 
modal is that the corresponding polynomial is real-rooted. When t = n — 1 
and t = 1 we get the Eulerian and the Narayana polynomials respectively. 
These are known to be real-rooted and Bona conjectures that the same is 
true for general t. In what follows we will prove the conjecture for t = 2. 

Let W be the set of finite words on N without repetitions. If w is any 
nonempty word we may write it as the concatenation w = LnR where n is 
the greatest letter of w and L and R are the subwords to the left and right 
of n respectively The stack-sorting operation s : W — > W may be defined 
recursively by 

J w, if w is the empty word, 
)s(L)s(R)n, if w = LnR is nonempty 



s(w) 



The stack sortable permutations in <S n are the permutations which are mapped 
by s to the identity permutation. Similarly, a permutation is called t-stack 
sortable if s*(7r) is the identity permutation. 

2. Stack sortable permutations and Jacobi polynomials 

The number of stack sortable permutations of length n with k descents 
are known ^H] to be the famous Narayana numbers ^2 Q2| 

The Narayana polynomials W n ^(x) are known to have real zeros. A sim- 
ple proof of this fact is obtained by expressing W n ^(x) in terms of Jacobi 
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polynomials. Recall the definition of the hypergeometric function 2-F1: 
2 F 1 (a,6;c;,) = ^ , 

n=0 

where (a)o = 1 and (a) n = a(a + 1) • • • (a + n — 1) when n > 1. The Jacobi 

polynomial P^^\x) can be expressed in the following two ways Page 
254]: 

pM) {x) = { ^±^ 2 F l (-n, 1 + a + P + n; 1 + a; 1^) , (1) 

i^(,) = ^(«±l)V(^-/f-n;l + a;^). (2) 

Rewriting W\(n + l,k) we end up with 

W n+ i tl (x) = 2 Fi (-n, -n - 1; 2;x) , 

which by gives 

WW*) = r^rC 1 - *) B ^ 1,1} f ^) • 

n + 1 \1 — x J 

Since the Jacobi polynomials are orthogonal when a,/5 > —1 we know that 
W n ,i(x) is real- and simple-rooted and that the zeros of W n ^(x) strictly 
interlace the zeros of W n+ i^(x), that is, {W n+ i j i(x)}^ =0 form a Sturm se- 
quence. 

The numbers W2(n, k) are surprisingly hard to determine despite of their 
compact and simple form. It was recently shown that 

(n + fc)!(2n-fc-l)! 
21 ' ' (fc + l)!(n-fe)!(2ife + l)!(2n-2A;-l)!' 

See [HI EH IE] for proofs and more information on 2-stack sortable permu- 
tations. 

A sequence of real numbers T = {"fk}k=o 1S cane d an n-sequence (of the 
first kind) if for any real-rooted polynomial / = oq + a\X + ■ ■ ■ + a n x n of 
degree at most n the polynomial := ao7o + a-i^ix + • • • + a n ^ n x n is 

real-rooted. There is a simple algebraic characterisation of n-sequences pj: 

Theorem 1. Let V = {7fc}^ =0 be a sequence of real numbers. Then T is an 
n-sequence of the first kind if and only ifT[(x + l) n ] is real-rooted with all 
its zeros of the same sign. 

We need the following lemma: 

Lemma 2. Let n be a positive integer and r a non-negative real number. 
ThenT = {(~Y)r k =0 is an n-sequence. 

Proof. Let r > 0. Then 



- E(-r)(D 



k=0 

2-F1 (— n, n + r; 1; x) 
P(°^ 1 )(l-2x), 
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where the last equality follows from Since the Jacobi polynomials are 
known to have all their zeros in [—1, 1] when a, /3 > — 1 we have that + 
l) n ] has all its zeros in [0, 1]. The case r = follows by continuity when we 
let r tend to zero from above. □ 

From the case r = in Lemma 13 and the identity 

e^y 1 )©^'- 1 '-^";) 

k=0 v / \ / k=Q \ / 

it follows that ( 2n ~ r 1 ) is an n-sequence. 

Theorem 3. For all n > the polynomial W2 t n(x), which records 2-stack 
sortable permutations by descents, is real-rooted. 



-x) n -\ 



Proof. We may write ^(n, k) as 



n-1 ) \n-l) \2k+l) 



n- 



A well known result on real-rooted polynomials reads as follows: If Y^i a i x% 
is a polynomial having only real non-positive zeros then so is the polynomial 
akiX 1 , where k is any positive integer. For a proof see 01 Theorem 3.5.4]. 
Applying this result to the polynomial x(l + x) 2n we see that J2 k Qfc+i) 2 ^ 
is real-rooted. Now, 

1 / n + k\ / 2n \ k ^-4 (In — k — 1\ / 2n \ n _i_fc 



E(n + k\( 2n \ k s -^(2n — k- 
\n-l)\2k + l) X n-1 



2/c + 1 



fc=0 v 7 v 7 k=0 

which by the discussion after Lemma is real-rooted. Another application 
of Lemma El gives that W n fl[x) is real-rooted. □ 
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